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ABSTRACT 



Method and system for command and control among a 
plurality of devices via a network by: connecting a first 
device to the network; connecting a second device to the 
network, where the second device stores application inter- 
face description data in a structured format for commanding 
and controlling the second device by other network devices; 
providing the application interface description data to the 
first device over the network; and sending control and 
command data from the first device to the second device 
over the network utilizing the apphcation interface descrip- 
tion data to control the operation of the second device. 
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Table 2: Attributes Table 
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Table 3: Example Formats and Types 



Type 


Format 
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CAL, AV/C,X-10 
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text 


html, plain 


audio 


wave, aiff 


application 


msword, pdf, 
postscript, gzip 
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METHOD AND SYSTEM FOR DEVICE TO 
DEVICE COMMAND AND CONTROL IN A 
NETW^ORK 

CROSS-REFERENCES TO RELATED 
APPLICAHONS 

Applicant claims the benefit of U.S. Provisional Applica- 
tion No. 60/084,578 entitled "Command and Control Using 
XML", filed on May 7, 1998 which is incorporated herein by 
reference. The present application is related to the following 
copending applications that are commonly assigned and 
which are incorporated herein by reference: U.S. patent 
application Ser. No. 09/104,299, entitled "Browser Based 
Command and Control Home Network" now U.S. Pat. No. 
6^3,716; U.S. patent application Ser. No. 09/104,297. 
entitled "Method and Apparatus for A Home Network Auto- 
Tree Builder" still pending; U.S. patent application Ser. No. 
09/104,298, entitled "Improved Home Network, Browser 
Based, Command and Control" now U.S. Pal. No. 6,198, 
479; U.S. patent application Ser. No. 9/103,469, entitled 
"Method and Apparatus for Creating Home Network Mac- 
ros" now U.S. Pal. No. 6,243,707; and U.S. patent applica- 
tion Ser. No. 09/104,606, entitled "Programming Tool For 
Home Networks now U.S. Pal. No. 6,182,094. 

HELD OF THE INVENTION 

The present invention relates to the field of network 
systems, and more particularly, to home network having 
multiple devices connected thereto. 

BACKGROUND OF THE INVENTION 

A network generally includes a communication link and 
various devices with communication capabihty connected to 
the communication link. The devices include computers, 
peripheral devices, routers, storage devices, and appliances 
with processors and communication interfaces. An example 
of a network is a home network for a household in which 
various devices are interconnected. A usual household can 
contain several devices including personal computers and 
home devices that are typically found in the home. As such 
the term "device" typically includes logical devices or other 
units having functionality and an ability to exchange data, 
and can include not only all home devices but also general 
purpose computers. Home devices include such electronic 
devices as security systems, theater equipment, TVS, VCRs, 
stereo equipment, and direct broadcast satellite services or 
(DBSS), also known as digital satellite services (DSS), 
sprinkler systems, lighting systems, micro waves, dish 
washer, ovens/stoves, washers/dryers, and a processing sys- 
tem in an automobile. 

In general, home devices are used to perform tasks that 
enhance a homeowner's life style and standard of living. For 
example, a dishwasher performs the task of washing dirty 
dishes and relieves the homeowner of having to wash the 
dishes by hand. A VCR can record a TV program to allow 
a homeowner to watch a particular program at a later time. 
Security systems protect the homeowner's valuables and can 
reduce the homeowner's fear of unwanted entry. 

Home devices, such as home theater equipment, are often 
controlled using a single common control unit, namely a 
remote control device. This single common control unit 
allows a homeowner to control and command several dif- 
ferent home devices using a single interface. Thus, may 
manufacturers have developed control units for controlling 
and commanding their home devices from a single interface. 
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One drawback associated with using a remote control unit to 
command and control home devices is that it provides static 
and command logic for controlling and commanding each 
home device. Therefore, a particular remote control unit can 

5 only control and command those home devices for which it 
includes the necessary control and command logic. 

In conventional network systems a user provides com- 
mands using a remote control unit or device control panel. 
Once the user ceases, there is no controller unit or device in 

10 the network to provide commands for automatic operation. 
For example, a home network can include a mner unit and 
a VCR without a tuner therein. After the user programs the 
VCR for time delay recording, a mechanism is necessary for 
the VCR to later automatically locate the tuner over the 

15 network and control the tuner to provide video information 
for the VCR to record. As such, after a user initially controls 
and commands a first set of devices, conventional systems 
do not provide a mechanism for the first set of devices to 
automatically communicate with a second set of devices in 

20 the network as necessary in order to accomplish tasks 
without direct user control and command of the second set 
of devices. 

There is, therefore, a need for a method and a system 
which provides dynamic and central control and command 
of devices in a home network. There is also a need for such 
a method and system to provide the ability for a user to 
initially control and command a first set of devices to 
communicate with each other and for the first set of devices 
to automatically communicate with a second set of devices 
in the network as necessary in order to accomplish tasks 
without direct user control and command of the second set 
of devices. There is also the need for such a method and 
system to provide the abilily for various network devices to 
automatically command and control other various network 
devices, 

SUMMARY OF THE INVENTION 

The present invention satisfies these needs. In one 
embodiment the present invention provides a method and 
system for command and control among a plurality of 
devices via a network by: connecting a first device to the 
network; connecting a second device to the network, where 
the second device stores application interface description 

45 data in a structured format for commanding and controlling 
the second device by other network devices; providing the 
application interface description data to the first device over 
the network; and sending control and command data from 
the first device to the second device over the network 

50 utilizing the application interface description data to control 
the operation of the second device. 

At least a portion of the application interface description 
data can be transferred to the first device for use, or the first 
device can query the application interface description data in 

55 other devices via the network. Further, application interface 
description data can be stored in a data base for the first 
device to access. The application interface description data 
can include remote procedure call information for the first 
home device to control the operation of the second home 

60 device. In addition, the application interface description data 
can include capabilities data for identifying the capabilities 
of the second device. The structured format for the appli- 
cation interface description data can include XML format. 
Preferably, a plurality of devices are connected to the 

65 network and each device contains application interface 
description data in the structured formal for commanding 
and controlling of the device by one ore more other devices 
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connected to the network. Id that case, the application 
interface description data of two or more of the plurality of 
devices can be provided to the first device via the network, 
so that the first device can send control and command data 
from the first device to the two or more devices over the 
network utilizing the application interface description data 
corresponding to each of the two or more devices to control 
their operation. 

In another aspect, the present invention provides a method 
and system for performing a service via a home network by: 
connecting a client device to the home network, wherein the 
chent device is capable of displaying user interface data; 
connecting a first home device to the network, where the first 
home device contains user interface data in a selected format 
that defines a user interface for user command and control of 
at least the first home device; connecting a second home 
device to the network, where the second home device 
contains application interface description data in a structured 
format for device command and control of the second home 
device; receiving the user interface data of the first home 
device at the client device; displaying the user interface 
defined by the user interface data of the first home device on 
the client device; accepting user input from a user in 
response to the user interacting with the user interface 
displayed on the client device; and sending control and 
command data from the client device to the first home device 
based on the user input to cause the first and second home 
devices to communicate with each other utilizing the appli- 
cation interface description data to perform the service. 

As such, the user can select the first and the second home 
devices from the user interface displayed on the client 
device. Further, the first home device can control the second 
home device by sending control and command information 
to the second home device utilizing the application interface 
description data based on the user input to the first home 
device via the client device. The application interface 
description data can includes capabilities data for the second 
home device, and the first home device can query the 
capabilities data within the application interface description 
data of the second home device, and update the user inter- 
face data in the first home device to allow commanding and 
controUing of the second home device by a user via the user 
interface of the first home device displayed on the client 
device. 

1\vo or more home devices can be connected to the 
network, each home device storing application interface 
description data in the structured format for commanding 
and controlling of the home device by one or more other 
home devices connected to the network. ITie application 
interface description data in each of the two or more home 
devices can include capabilities data for the respective home 
device. In that case, the first home device can query the 
capabilities data within the application interface data of the 
two or more home devices, and update the user interface 
data in the first home device to allow commanding and 
controUing of the two or more home devices by a user via 
the user interface of the first home device displayed on the 
client device. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other features, aspects and advantages of the 
present invention will become better understood with regard 
to the following description, appended claims and accom- 
panying drawings where: 

FIG. 1 shows a block diagram on an embodiment of a 
network according to one aspect of the present invention; 
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FIG. 2 shows the block diagram of FIG. 1 in an example 
device control and communication scenario; 

FIG. 3 shows a block diagram of an example home 
network system according to the present invention which 
includes a plurality of client and server devices; 

FIG. 4 shows a block diagram of example embodiments 
of a client device and a server device of FIG. 3; 

FIG. 5 shows example embodiments of client devices; 

FIG. 6 shows example embodiments of server devices; 

FIG. 7 shows a block diagram of two example networked 
server devices capable of communication with, and control 
of, one another; 

FIG. 8 shows a block diagram of an example architecture 
of an audioA^ideo (AAO model including examples of a 
source server device, a sink server device and a client device 
in a network; 

FIG. 9 shows another example audio/video (AAO model; 

FIG. 10 shows an example capabilities data table for a 
network device; 

FIG. 11 shows an example attribute data table for a 
network device; 

FIG. 12 shows an example configuration of building 
blocks for generating command messages among networked 
devices; 

FIG. 13 shows another example configuration of the 
building blocks of FIG. 12 for generating command mes- 
sages; 

FIG. 14 shows three examples of interaction among 
networked cfient and server devices; 

FIG, 15 shows an example block diagram for definitions 
of API extensions of networked device interfaces; 

FIG. 16 shows an example architecture for a server device 
application accessing the interface description document of 
another server device; 

FIG. 17 shows another example inter-device control 
architecture between a controller server device and a con- 
trolled server device; 

FIG. 18 shows an embodiment of an XML protocol 
providing a Web standard common middleware layer in a 
communication stack at the API level between networked 
devices; 

FIG. 19 shows another embodiment of server device to 
server device command and control architecture; 

FIG. 20 shows the relationship between a device interface 
library and consumer electronics definition data base for 
home devices; 

FIG. 21 shows hierarchal form of an embodiment of a 
device interface definition; 

FIG. 22 shows an example of layers in device interface 
definition of FIG. 21; 

FIG. 23 shows a command transmission and interpreta- 
tion process between a sender and receiver device; and 

FIG. 24 shows an example table of a partial Ust of packet 
types and formats for providing translation services accord- 
ing to an aspect of the present invention. 

DETAILED DESCRIPTION OF THE 
INVENTION 

In one aspect, the present invention provides inter-device 
communication in a network such as a home network. As 
home devices become more intelligent and can share 
information, inter-device communication allows a user to 
interconnect devices in a network to take advantage of the 
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information sharing capabilities of those devices. As such, for displaying a GUI 18 using a GCO 22 stored in the client 

inter-device communication plays a crucial role in affording device 12 or transferred to the client device 12 over the 

a user with the ability to fully and flexibly utilize the network from a desired server device 14. For example, in an 

networked devices. initial device selection phase, the client device 12 can fetch 

Referring to FIG. 1, in an embodimem of the present s "'^ ^2 of at least one se>yer device 14 over the 

invention, a network 10 includes at least one cUenl device 12 "Tr^'^^'f f T .k ^^a^' ' i . L Ik' 

... , . 1 ^ ' . . J • GCO 22 for controlung the server device 14. Preferably, the 

and at least one server device 14 interconnected via a ^UI 18 is customized to the server device 14 and can 

comrnunication Imk 16. The communication hnk 16 can j^^j^^^ ^ ^^j^.j^ command set for controlling the server 

mclude a 1394 serial bus providing a physical layer device 14 

(medium) for sending and receiving data between the van- lo '.^^^ ^^^^ ^^.^^ ^^^^ ^^^.^^ 
ous connected home devices. The 1394 serial bus supports ^^^1^^^ commonalities such as: (1) a common GCO model 
both time-multiplexed audio/video (AAO streams and stan- (yp^ fo^ client device tenderer 24 to display GUIs 18, (2) 
dard IP (Internet Protocol) communications. In certam common communication protocols for transferring the 
embodiments, a home network uses an IP network layer as qcOs 22 from various server devices 14 to the client device 
the communication layer for the home network. However, 15 and (3) common communication protocols for GUI 
other communication protocols could be used to provide interaction from the client device 12 to the control program 
communication for the home network. 20 of the corresponding server device 14. wherein the chent 
Each client device 12 may communicate with one or more device 12 does not require a built-in knowledge of a par- 
server devices 14 in the network 10. Further, each server ticular server device 14 being controlled, 
device 14 may communicate with one or more other server Referring still to FIG. 4, a server device 14 may include 
devices 14, and one or more client devices 12, in the network one or more server control programs 20 to control the server 
10. Each client device 12 can include a user communication hardware for providing a service. The GUI interface 18 from 
interface including input devices such as a mouse and the GCO 22 of the server device 14 provides interface to the 
keyboard for receiving user input, and a display for provid- server device control programs 20. The server device 14 
ing a control user interface for a user to interact with the may also include control state data 26 indicating the control 
networked devices. The user interface can include a graphi- status of the server device 14 and server device hardware in 
cal user interface (GUI) display 18 for providing informa- providing a requested service. 

tion to the user. Referring to HG. 2, as defined herein, each p^^ example, the control slate data 26 can include the 

server device 14 provides a service for the user, except status of control information in the GUI 18 for the server 

control user interface, and each client device 12 provides d^^i^e 14, such as timer setup for a recording action in a 

control user interface for user interaction with the network yCR server device. The control state data 26 is stored in the 

10. As such, only client devices 12 interact directly with controlled server device 14, and displayed to a user through 

useni, and server devices 14 mteract only with client devices t^e GUI 18 of the server device 14 at the controlling client 

12 and other server devices 14. Example services can ^j^vice 12. for user control of the server device 14. 

include MPEG sourcing/sinking and display services. Preferably, the controlling client device 12 for displaying the 

FIG. 3 shows a block diagram of an example home GUI 18 of the server device 14 does not retain knowledge of 

network 10 that includes a plurality of client devices 12 and the control state data 26 for the controlled server device 14. 

a plurality of server devices 14. Each server device 14 may Each server device 14 can be conUx)Ued by one or more 

include hardware as a resource in the network for providing client devices 12. As such, the control state data 26 stored in 

services to the user. Further, each server device 14 may store t^e server device 14 includes status of the information in the 

a server or service control program 20 for controlling the GUI 18 of the server device 14 at each of the controUing 

server hardware, and include a graphical control object ^Uent devices 12. For example, when the user controls a 

(GCO) user interface description 22 for user interface with server device 14 using a first client device 12. upon comple- 

the server control program 20 as shown in FIG. 4. ^^^^ control, the information in the GUI 18 of the 

For conUxDl between a controlling client device 12 and a server device 14 at the first client device 12 is saved by the 

controlled server device 14, the client device 12 accesses the server device 14 in the control state data 26 of the server 

GCO 22 of the server device 14 by, for example, transferring device 14, 

the GCO 22 from the server device 14 to the client device Alternatively, while the user is interacting with the GUI 
12 over the network. The client device 12 then uses the 50 ig of the server device 14 at the first client device 12, the 

transferred GCO 22 to create a control user interface GUI 18 control slate data 26 of the server device 14 is updated with 

for the user to communicate with the control program 20 of the information in the GUI 18 of the server device 14 at the 

the server device 14 from the client device 12 over the first client device 12, and upon completion of user control, 

network. The user provides command and control to at least the control state data 26 is retained in the server device 14. 
the control program 20 of the server device 14 firom the 55 when the user controls the server device 14 using a second 

client device 12. client device 12. the control stale data 26 is made available 

Storing the GCO 22 of each server device 14 in the server to the user via the GUI 18 of the server device 14 at the 

device itself may reduce the processing and storage require- second client device 12 for further control. The user can also 

meats of the client devices 12 in networks with several use the first chent device 12 at a later time to control the 
server devices 14. Further, storing the GCOs 22 in the server 50 server device 14, whereupon the control state data 26 is 

devices 14 may allow each server device 14 to provide its made available to the user via the GUI 18 of the server 

own GUI look and feel to the user, and allows for modifi- device 14 at the first client device 12 for further control. The 

cation or updating of the GCOs 22 without modifications to server device 14 can also include a clock 28, or maintains the 

client devices 12. current lime, to allow time delay action based on time or 

Referring to FIG. 4, to provide command and control 65 clock input from a user, as described below, 

between a client device 12 and the server device 14, in one A client device 12 and a server device 14 can be physi- 

embodiment, the client device 12 can include a renderer 24 cally bundled together as one unit such as a DTV. In that 
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case, the server device 14 includes a control program 20 for for controlling data stream sink hardware 34 of the sink 

controlling the server hardware, and the client device 12 server device 14. In an example operation, a user utilizes the 

provides control user interface to the server control program client device 12 to control the source server device 14 to 

20 for control and command of at least the server hardware. start the data stream source hardware 32, and lo control the 

FIG. 5 shows examples of client devices 12 that may 5 sink server device 14 to start the data stream sink hardware 

include: (1) a PDA(RemoteC) for displaying a GUI, (2) a 34. Upon initiation of data transfer from the data stream 

DTV(STB) for displaying a GUI and including a sink server source hardware 32 to the data stream sink hardware 34, the 

comprising audio and/or video program streak destination user can relinquish the client device 12. Alternatively, the 

server, and (3) a PC for displaying a GUI and including at user can program the initiation of the data transfer for a 

least one server device for providing multiple services. future time and relinquish the client device 12. Thereafter, 

Hardware and execu tables in a DTV or PC client device can the data stream source hardware 32 of the source server 

also be controlled by other client devices. FIG. 6 shows device 14 and the data stream sink hardware 34 of the sink 

example server devices 14, including: (1) a DVDP Smart- server device 14 automatically initiate the data transferal the 

Card as a source server device, (2) an Audio AmpUGer as a time programmed by the user. 

sink server device, (3) a DVCR as either a source or a sink For example, the data stream source hardware 32 can 
server device, and (4) a Management Server for managing include a Tuner- Access Device such as a Direct Broadcast 
remote server devices. The Management Server can be Satellite (DBS). A DBS is a multi-channel alternative to 
included in a DBS-STB, Cable TV-STB, or ATSC-STB, for cable television and provides cable-like television program- 
example. Such devices include a Management Server for ming directly from satellites on small (18 inch to 3- foot 
local control or management of the internal workings of the diameter) satellite dishes. With DBS, several standard ana- 
STB. Further, external servers accessed through an external log television signals are digitally compressed onto a single 
network can be utiliiied by local client devices for services sateUite transponder thereby allowing up to 200 or more 
such as Video-on-Demand, Enhanced-TV, and Internet channels receivable with a dish pointed at a fixed position in 
commerce, for example. the sky. The data stream sink hardware 34 can include a 

Referring to FIG, 7, communication and control between 25 Digital Video Cassette Recorder (DVCR) which comprises 

two server devices 14 is accomplished by the control pro- s digital VCR that is able to decode compressed digital video 

grams 20 of the server devices 14 communicating command signals on playback. The user provides command and con- 

and control data therebetween. A server device 14 can trol data including "time-delay record" event data for the 

control one or more other server devices 14 over the DVCR and a "time -delay select a program" event data for 

network. And, a server device 14 can be controlled by one 3Q the Tuner- Access Device. After the time-delay, the Tuner- 

or more server devices 14, and by one or more client devices Access Device selects the desired program, and sources 

12. Further, a user can utilize a client device 12 to control program data to the DVCR which receives and records the 

and command a first set of server devices 14, and the first set program data without further control actions from the user, 

of server devices 14 can automatically command and control FIG. 9 shows another example audioA^ideo (AAO model 

a second set of server devices 14 without user involvement, 35 including at least a source server device 14 SERVERl, a 

as necessary to perform services to the user. sink server device 14 SERVER2 and a client device 12 in the 

For example, for automatic time -de lay operation, a user network 10. The client device 12 includes a session manager 

can "log on" to a client device 12 lo control a first set of 36 with a user interface for displaying selection information 

server devices 14 and specify desired services. The user then for a user to select and control the server devices 14 

"logs off" from the client device 12. The first set of server 40 SERVERl, SERVER2 and other server devices 14 such as 

devices 14 perform communication and control among SERVER3 and SERVER4 (not shown). The selection infor- 

themselves, and at a later time, one or more of server devices mation can include iconic symbols designated as Servl, 

14 in the first set automatically control a second set of server Serv2, Serv3 and Serv4 in the session manger 36 for a user 

devices 14 as necessary to collectively provide the desired to select the server devices 14 SERVERl, SERVER2, 

services without user involvement, 45 SERVER3 and SERVER4, respectively. The source sever 

FIG, 7 shows example embodiments of two server device 14 SERVERl can include a DVCR and the sink 

devices 14 capable of communication with, and control of, server device 14 SERVER2 can include a ViDTV, 

one another. Each server device 14 includes a control In one example operation, upon selection of the server 

program 20, a clock 28 and control state data 26 described devices 14 SERVERl and SERVER2, the client device 12 

above. Each server device 14 can also include a GCO 22 for 50 transfers the GCO 22 of each server device 14 to the client 

the server device 14 to be directly controlled by a client device and displays a corresponding GUI 18 for each of the 

device 12. However, a GCO 22 does not need lo be included server devices 14 SERVERl and SERVER2, The user can 

in a server device 14 that is not directly controlled by a client interact with the GUI 18 of each server device 14 lo provide 

device 12 and only communicates with other server devices command and control to the corresponding server device 14 

14, Each server device 14 also includes a command Ian- 55 for service. Each server device 14 can provide service alone 

guage (CL) interface 30 and a library of commands. The or in combination with other server devices 14. Further, the 

library of commands includes the commands that the server session manager 36 transfers control state data 26 between 

device 14 utilizes to send and receive information for the GUIs 18 of the server devices 14 in the client device 12 

providing its service. However, a command language is not as necessary for the corresponding server devices 14 to 

necessary for user control as shown in FIG. 4 and described 60 perform a service. Based on the user command and control 

above. information, two or more of the server devices 14 can 

FIG. 8 shows an example audio/video (A/V) model communicate command and control information therebe- 
including a source server device 14, a sink server device 14 tween to provide a user requested service, 
and a client device 12 in the network. The source server The session manager 36 can include a software agent 
device 14 includes a control program 20 for controlling data 65 which functions to access and display available home net- 
stream source hardware 32 of the source server device 14, work services provided by various server devices 14 in the 
and the sink server device 14 includes a control program 20 network 10. The software agent can additionally match the 
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capabilities of various server devices 14 in the network 10 
and display selection information for only those server 
devices 14 that have compatible capabilities. Further, the 
session manager 36 can match the selections made in the 
GUI 18 of one server device 14 to the selections in GUI 18 5 
of another server device 18 to help the user provide mean- 
ingful command and control information to the server 
devices 14. 

In another example operation, the session manager 36 
executes the software agent which searches the network and 
discovers the server devices 14 connected to the network. 
The software agent also accesses capabilities data stored in 
each server device 14 to determine the capabilities of the 
server devices 14 and provide information about those 
capabilities to the user. The session manager 36 then dis- 15 
plays the selection icons Servl, Serv2, Serv3 and Serv4 for 
the server devices SERVERl, SERVER2, SERVER3 and 
SERVER 4 as shown in FIG. 9. 

The session manager 36 initially enables all the selection 
icons Servl, Serv2, Serv3 and Serv4 to allow the user to 
select from among all four selection icons. After the user 
selects the server device SERVERl by clicking on the Servl 
selection icon, the session manager 36 determines that the 
server devices SERVER3 and SERVER 4 are incompatible 
in capability with the server device SERVERl. As such, the 
session manager 36 disables the selection icons Serv3 and 
Serv 4 for server devices SERVER3 and SERVER4, respec- 
tively. The user can then click on the icon Serv2 to command 
and control the server device SERVER2. 

As the user interacts with the GUI 18 of a selected server 
device 14, control and command information input by the 
user into each GUI 18 provide additional capabilities infor- 
mation which affect further server device selections by the 
user. For example, if a VCR server device 14 is selected, 
further action by the session manager 36 in enabling or 
disabling selection icons for other server devices 14 is 
affected by a user decision to play or record. 

Each server device 14 in the network has one or more 
service capabilities as discussed above by way of example 
with reference to the server devices in FIG. 9. Each service 
capability includes sourcing or sinking of information. For 
example, a TV has the sinking capability of receiving video 
and audio streams, a VCR device can source (transmit) and 
sink (receive) video and audio signals, and a PC may be able 
to transmit and receive video, audio and data. Each sourcing 
capability has a complementing, and compatible, sinking 
capability. Similarly, each sinking capability has a 
complementing, and compatible, sourcing capability. For 
example, a video output capability of one device is comple- 
meoted by a video input capability of another device. 

Since each device 14 can be a source or sink for several 
different services on the network, each device 14 stores a 
capabilities data table (Capabilities Table 1) as shown by 
example in FIG. 10. The first column of Table 1 identifies the 55 
service capabilities of a device 14, and the second column 
identifies whether the device 14 is a source or a sink for a 
corresponding service in the first column. Using the capa- 
bilities data Table 1, new services can be implemented while 
maintaining compatibility with older devices. For example, go 
if a new service is developed that is compatible with an older 
service, both the new and the old service can be entered into 
the capabilities data Table 1 for a device implementing the 
new service, whereby the implementing device remains 
compatible with older devices using the old service. 55 

In one implementation, a Device Manager conducts a 
matching or comparison of device source and sink services. 
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For example, the Device Manager can be implemented as a 
software agent to compare the capabilities or properties of 
various devices 14 and locate devices 14 with matching 
capabilities. For example, in a case where the service is a 
media stream from a first device 14 across the network to a 
second device 14, the Device Manager compares the capa- 
bilities of the first and second devices 14 to assist the user 
in making a sensible selection of the second device 14 which 
is compatible with the capabilities of the first device 14. The 
following is an example list of service capabilities for an 
embodiment of a server device 14: 

Stream_format_video_dv 

Stream_format_video_mpeg2tpt 

Stream_format„video__dsstpt 

Stream_format_video_mpeg2pes 

Stream_format_video_mpeg210801-tpt 

Each device 14 can further store an attribute data table 
(Attribute Table 2) including pertinent attributes of the 
device, shown by example in FIG. 11. A name and a value 
define each attribute within Table 2. Though character 
lengths are shown in Table 2, they are not required. The 
attribute data is available to other devices 14 on the network 
10 to facilitate interoperability and to store device informa- 
tion. For example, a Device Page as described below uses 
the Attribute Table 2 to store the device name. Other fields 
can be added to the attribute data Table 2 as necessary. 

In the user-to-client device control model described 
above, attribute data can be displayed on the GUI page of the 
server device 14 at the client device 12. Alternatively, a 
second level device information home page can be utilized 
to display said attribute data. Further, the attribute data in the 
form of a text or Extensible Markup Language (XML) file 
can be accessed by a software agent. For the device-to- 
device control model, the attribute data for the controlled 
device is stored in the device interface application interface. 

The Device Location attribute field in the Attribute Table 
2 is used to store the location or group for each device 14. 
The Device Type attribute field specifies the device type, 
such as VCR, DVD, DTV, Camcorder, PC, Security System, 
etc. for the particular device 14. The Device Type attributes 
field is used to select a default device icon to represent the 
device within the Device Page if the device itself does not 
supply one. The Attribute Table 2 can include multiple 
entries for the Default Source and the Default Sink attributes 
fields. Each such entry represents a different default source 
or sink device 14 for each data type handled by the device 
14. 

Preferably, the capabilities and attributes data are pack- 
aged into structured data using a hierarchical language. This 
provides a common method of retrieving the capabilities and 
attributes data that are used for other purposes such as in 
GCO transfer and server device-to-server device control. As 
an example, the attributes data can include the following 
strucmred data format: 

<DEVICEATTRIBUTES> 

<ATTRIBUTE name-DeviceManufacturer value- 

"Samsung Inc,"> 
<ATTRIBUTE name^Manufacturer URL value- 

www.samsung.com> 
<ATTRIBUTE name-Manufacturerlcon value- 

"logo.gif'> 

<ATTRIBUTE name-DeviceName value-"Samsung 
DSS"> 

<ATTRIBUTE name-DeviceModel value-"SCH1900"> 
<ATTRIBUTE name-DeviceType value-DDS> 
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<ATTRIBUTE Dame»DeviceLocation value« 

"lJvingroom"> 
<AJ TRIBUTE name=DeviceIcon value-"device.gif *> 
<ATTRIBUTE nameaDeviceAddress value« 

1 05,144-30. 17> ^ 
</DEVICEAnrRIBUTES> 

As an example, the capabilities data can include the 
following structured format: 

<DEVICECAPABILITIES> 

<CAPABILITY type«MPEG2 value=Source> 

<CAPABILITY type=MPEG2 value=Sink> 

<CAPABILITY type=MPEG3 value=Source> 

<CAPABILITY typeoMPEG3 valueoSink> 

</DEVICECAPABILITIES> 15 

An application interface language is utilized to allow 
different server devices 14 to perform device-to-device 
control, including sever device-to-server device control. The 
application interface language includes command 
languages, and can be described using XML, as detailed 20 
below. The control program 20 of one server device 14 
remotely controls the control program 20 of another server 
device 14 over the network, without using GUIs 18 or user 
involvement. An example of device-to-device control is 
automatic operation. A user initially provides control 25 
through a client device 12 for a desired service, and subse- 
quently two or more server devices 14 automatically com- 
municate and control one another without further user 
interaction to provide the service. 

Referring to FIGS. 12 and 13, preferably a standard 30 
application interface language is utilized to allow interop- 
erability among various control programs 20 in various 
server devices 14. In one embodiment, the standard appli- 
cation interface language includes the following building 
blocks: (1) functional specification of service 40 such as in 35 
a service function database, (2) a block where elements of a 
message are composed 42, (3) industry standard format 44, 
(4) message compression 46, and (5) message string con- 
struction 48 to output structured message data. 

FIG. 12 shows an example configuration of the building 40 
blocks to perform the function of generating command 
messages. Each message item is composed from the func- 
tional specification of service and standardized by selecting 
an industry standardized compressed form (Hex) label for 
the message item, A group of such message items are 45 
assembled to create a complete command string. Existing 
command languages such as CAL and AV/C operate as 
shown in FIG. 12, However, such command language 
mechanisms specify binary or hex code messages and sys- 
tem operation on physical devices on the physical interface, 50 
and are based on hardware specifications. Therefore, such 
command languages may be less desirable for a network 
layer based control mechanism where a control system 
specification includes naming, addressing, device capability 
discovery, communication language and command mes- 55 
sages at the application level software level, where one 
software application program 20 in a controller device 14 
locates and controls another software application 20 pro- 
gram in a controlled device 14 over the network 10. Said 
control mechanism is more suitable for devices such as 60 
digital appliances including appliances (e.g., DVCR) as well 
as multi-purpose, multi-application devices such as comput- 
ers. 

FIG. 13 shows a preferred example configuration of the 
building blocks of FIG. 12 to perform the function of 65 
generating command messages. In FIG. 13, the positions of 
the industry standard format 44 and the message compres- 
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sion 46 are different than in FIG. 12. A number of textual 
standard forms are selected from the functional specification 
service 40 to make a complete message. Later the message 
may be compressed by a lower layer of the protocol stack. 
FIG. 13 represents a method of periforming service or device 
command and control for consumer electronics (CE). Mes- 
sage composition can be defined by the XML standard 
syntax and compression can be performed by another pro- 
tocol layer such as HTTP. A command interface language is 
utilized at the application software 20 interface level, rather 
than lower hardware levels. As such, the network protocol 
stack is governed by commands in said language, and each 
of a controller device 14 and controlled device 14 can be 
viewed as integrated components of the network for mes- 
sage transmission therebetween. 

Referring to FIG. 14, three different instances of interac- 
tion among client devices 12 and server devices 14 are 
shown. In the first instance "A", a human user communi- 
cates with a remote service application "S". The user utilizes 
a browser in a client device 12 as the user interface, wherein 
the browser controls service programs 20 in the service 
application "S" and receives response in Hyper Text Markup 
Language (HTML) or XML formats. A secondary server is 
included with the browser to accept XML based asynchro- 
nous command message postings. For example, for a DVCR 
the secondary server 14 can accept command messages such 
as "VCR FAILED: TAPE BROKE." A software agent 
including a browser can be utilized to display the command 
messages for a user in the browser's GUI for later attention 
by the user and control of the DVCR. Preferably, an XML 
based client device 12 includes an HTTPl.l server capabil- 
ity to respond to command initiated elsewhere for server 
device to server device command and control. 

In the second instance "B", the user is replaced by a 
software client control program 50. The software client 
control program 50 generates XML based command post- 
ings to the service application "S" and receives back XML 
command postings. And, in the third instance **C", the 
software client control program 50 is replaced by an appli- 
cation such a server device control program 20, wherein 
commands and responses are exchanged between two ser- 
vice applications 20. In that regard, instance "B" is a special 
case of instance "C** with a null service. 

An application interface language based on XML is used 
for control between a first server device 14 and a second 
sever device 14 (device-to-device or service-to-service) for 
devices or services that are world wide web (Web) enabled 
and Internet enabled. The application interface language is 
based on the Web standard, middleware layer. In one 
embodiment, device-to-device control includes remotely 
controlling the control program 20 or Application, in one 
server device 14 from another server device 14 in the 
network 10. As such, the interfaces (API) to such Applica- 
tions 20 are made available over the network using API 
extensions. Preferably, the API extensions utilize a standard 
format, such as an XML-based interface, to provide overall 
interoperability. 

Referring now FIG. 15, there is shown block diagram 
definitions of API extensions for a first Application A, 
designated as Service A, and a second Application B, 
designated as Service B, communicating over the network 
10, For example, the Service A can be the control program 
for a first server device A in the network, and the Service B 
can be the control program for a second server device B in 
the network. The server device B sends commands to the 
server device A. For this example, the first and second 
service devices A and B can include CE devices. 
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Referring to the API extensions for the Service A, the first XML form to comply with the Web/Internet standard XML 
upper-most block 52 provides a comprehensive definition or for inter-device communication. 'ITie XML method calls 
data base of CE objects and methods using English words to (messages) are sent to Service A over the network, and 
describe CE devices. The comprehensive definition or data Service A reconverts the XML method calls from the net- 
base can also be in C, XML or other formats capable of 5 work interface to program code API definitions for Service 
representing objects and their respective methods. The com- A. This conversion and re-conversion provides Web/Interaet 
prchensive definition or data base utilizing XML is termed compatibility for diverse devices in the network with pro- 
XCE definition. The second block 54 provides a format for gram code APIs which would otherwise require binary 
representation of an API in XML form for all devices 14, compatibility between different devices, 
designated as an interface data type definition INTER- lo Appendix 1 shows examples of the XML interface blocks 
FACE.DTD. utilizing the block diagrams in of FIG. 15. 

At A software agent, designated as Tool A, utilizes a Further, Appendix 1 provides examples of interface defi- 

subset of the XCE definition for Service A. and uses the nitions INTERFACE.DTD and CALL.DTD used to create 

interface data type INTERFACE.DTD for Service A to description documents of available services, 

create an XML form document, INTERFACE-A.XML. The 15 INTERFACE.XML, described above. The CALL.DTD defi- 

document INTERFACE-A.XML describes the objects and nition includes a rule set for generating method call or 

methods supported by the Service A according to the docu- function call message such as XML Remote Procedure Call 

ment type definition INTERFACE.DTD for Service A. (RPC) or XMLRPC messages. The CALL.DTD definition 

Other data type definitions can also be used to create the describes an output interface of a controller service 14. In a 

INTERFACE-A.XML document. 20 home network, for example, INTERFACE.XML represents 

The software Tool A also creates a look-up table 56 to the services available on the home network. The available 

convert from XML messages from Service B on the network services are a subset of the entire services in the CE space, 

interface, to API definitions for Service A, programmed in C In a One -Touch- Record (OTR) scenario, a user is in 

for example, and complied to executable binary. Preferably, control of a Tuner- Access-Device such as a Satellite STB. 

the look-up table 56 is created at compile time, whereby 25 The user controls the tuning using an Electronic Program 

during run-time, incoming XML form method messages Guide (EPG) such as a graphical user interface representa- 

(commands) from Service B are converted to the API format tion of program listings. OTR record provides the user with 

created by the complied application C code for Service A. a service Aft including selection of a future program from 

The look-up 56 table provides run-time translation of XML the EPG for recording without the user accessing the VCR 

object method calls from Service B into device native 30 graphical user interface lo program the VCR for a Time 

language calls for Service A. The look-up table 56 is Delayed Recording. OTR automates the control of the VCR. 

complied with the device control program 20 for local Below is an example control list of actions in OTR.XML 

execution on the server device A for Service A. shown in Appendix 2: (1) StreamOpeD=play the selected 

The INTERFACE- A. XML can be used by Service A for program stream output to the network from a Satellite STB; 

validity checks if it encounters an error in a received 35 for OTR this control is local to the STB device; (2) 

message. The INTERFACE-A.XML can also be used by a SlorageOpen=open a storage service; and (3) 

foreign Application such as Service B to determine the StorageRecord^Send the Record command across the nel- 

message format for Service A before communicating with work to the VCR. 

Service A. Further, if a message from Service B to Service As discussed above in relation to FIG. 15, a first device B 
A causes an error. Service B can access the INTERFACE- 40 can access the INTERFACE.XML document of a second 
A.XML document to diagnose the error. device A to examine the device capabilities and API inter- 
Referring to the API extensions for the Service B, the first face details of the second device A and determine supported 
block 58 provides a comprehensive definition or data base of functionality and command details of the second device A. 
CE objects such as the XCE definition for Service A above. In particular, the first device B can determine overlapping, 
The next block 60 provides a language definition for making 45 and therefore useable, methods supportecl by first device B 
XML form method (command) calls to remote API services and the second device A. FIG. 16 shows an example wherein 
or devices such as the API for Service A. The language a first server device B including an Application B accesses 
definition is a document type definition Method Request the INTERFACE-A.XML document of a second server 
CALL.DTD which describes interaction with objects on the device A including an Application A. The first server device 
network. so B includes a INTER FACE-B. XML document for compari- 
A software agent, designated as Tool B, utilizes at least a son with that of a INTERFACE-A.XML document in the 
subset of the objects and methods in the XCE definition for second server device A. 

Service B and the CALL.DTD document, to generate a In one scenario, the first server device B wishes to control 
look-up table 62 for converting commands from a complied the second sever device A in the network. TTie INTERFACE- 
C program code for Service B into XML form method 55 A.XML document of the second device A is transferred from 
requests. As such, the look-up table 62 provides conversion the second server device A to the first server device B and 
between a method invoked by Service B (e.g., "PLAY") and used by Application B to query the capabilities and API 
the XML document or message that carries the method call interface methods of the second server device A. This allows 
across the network interface to Service A, for example. The the first server device B to control the second server device 
subset of the XCE definition used by software Tool B 60 A utilizing XML remote procedure calls XMLRPC. In 
depends on the extent and nature of use of the network. For another scenario, the first server device B performs the 
example, the subset can be selected to provide global or above steps after attempting to communicate with the sec- 
restricted use of all available services on a home network. ond server device A at least once, and failed to establish 
Therefore, the API extensions provide for communication communication. Yet in another scenario the first server 
between various devices on the network using XML. In the 65 device B queries the INTERFACE-A.XML document in the 
example above, the program code 20 for Service B generates second server device A remotely without transferring the 
method calls lo an API, and the API calls are converted lo INTERFACE-A.XML document lo the first server device B. 



03/31/2004, EAST Version: 1.4.1 



us 6,4( 

15 

Upon examining the contents of the INTERFACE- 
A.XML document, the first server device B can create 
commands for sending to the second server device A in 
XML format as described above. Generally, the first server 
device B can interpret at least a portion of the contents of the 
I NTERFACE-A.XML document that overlaps with a subset 
of the XCE definition used by the first and second server 
devices B and A as described above. If the first server device 
B is unable to interpret a portion of the contents of the 
I NTERFACE-A.XML document, then the first server device 
B can ignore that portion, or fetch an appHcation to assist it 
in interpreting that portion, by translation as described 
further below. 

Referring to FIG. 17, another example device-to-device or 
inter-device control between a controller server device 14 
and a controlled server device 14 is shown. The controller 
device 14 includes a controller application E and the con- 
trolled device 14 includes an application executable C. The 
controlled device 14 further includes INTERFACE-A.XML, 
the application interface description A of the application C. 
Application E accesses the application interface description 
A in the controlled device 14 to query the capabihties and 
API interface methods of the controlled server device 14. 
Application E then commands and controls appHcation C 
using XML remote procedure calls to control hardware or 
service D of the controlled device 14, A scheduler device can 
be a case of a controller device 14, driven by time of day 
such as Time -Delay-Record controller in a VCR. 

In a first example, the application E accesses the appli- 
cation interface description A by remote query over the 
network. In a second example, the appHcation E accesses the 
application interface description A by transferring a copy of 
the appHcation interface description A from the controlled 
device 14 to the controller device 14. The application E then 
queries the interface description A locally. In a third 
example, the application interface description A is trans- 
ferred to a library device 64 which provides library space for 
interface descriptions, and the application E remotely que- 
ries the interface description A in the library. The Hbrary 
device 64 stored the address (URI) of the associated appU- 
cations available for direct control action and responses. 

Referring to FIG. 18, the XML protocol provides a Web 
standard common middleware layer in a communication 
Slack 66 at the API level between applications of various 
devices 14 in the network. In each device 14, applications at 
the top of the communication stack send and receive com- 
munication messages over the network, and communicate 
with software layers in the device stack that locally control 
the device hardware or service software for the device. 

A first XML layer API, designated as XML Layer OUT 
68, is used for sending messages, and a second XML layer 
API, designated as XML Layer IN 70, is used for receiving 
messages. The XCE definition and the XML definition of a 
method call, namely the document type definition CALL- 
.DTD described above, are used to create the XML Layer 
OUT 68. Further the XCE definition and the XML definition 
for a method call, namely document type definition INTER- 
FACE.DTD described above, are used to create the XML 
Layer IN 70. For example a controller application utiHzes 
the XML Layer OUT 68 and a controlled appHcation uliHzes 
the XML Uyer IN 70. 

Referring to FIG. 19, another embodiment of server 
device-to-server device command and control architecture is 
shown. An XML-based control architecture is utilized for 
device-to-device (service to service) control for Web and 
Internet enabled devices or services. A first device A can 
remotely control an appHcation 20 in a second device B over 
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the network using XML command messages. The interface 
to each device includes interfaces to the applications in the 
device, and is described in XML format. Said interfaces can 
be extended and made available on the middleware layer for 

5 retrieval and interpretation by other devices over the 
network, as described further below. 

Each of the server devices A and B includes hardware and 
software for controUing other server devices over the net- 
work and for being controlled by other server devices over 

10 the network. In FIG. 19, the home network device A is a 
controUer device or module, and the home network device 
B is a controUed device or module. Each of the devices A 
and B includes a local Device XML Interface 72 comprising 
an interface document INTERFACE.XML and a document 

15 type definition INTERFACE.DTD. The INTERFACE.XML 
document includes a description of the objects, methods and 
parameters supported by the corresponding device 14. The 
INTERFACE.DTD document can be used for vaHdity 
checks specific to the XML interface of the device, as 

20 described above. 

Each of the devices A and B also includes an XML parser 
74, comprising program code for parsing and vaHdating 
XML messages, such as XML interface and XMLRPC 
commands. The XML parser 74 is similar to said XML 

25 Layer IN 70 described above with reference to FIG. 18. 
Further, each of the devices A and B includes an XMLRPC 
encoder and decoder (codec) 76 for encoding method names 
and parameters of an outgoing caU in an XMLRPC message, 
and for decoding an incoming XMLRPC message after it is 

30 parsed, to retrieve the method name and parameters therein. 
The XMLRPC codec 76 is independent of the device XML 
interface 72 and of the device-to-device control architecture, 
thereby allowing use of different XMLRPC formats without 
changing other aspect of the device to device control archi- 

35 lecture. 

An Interface Fetcher comprising program code, is utilized 
by each of the devices A and B to fetch the device interface 
of another device directly from another device or from a 
home network Interface Library 80. When a device 14 is a 

40 controller device, a controller appHcation program code 82 
in the controller device 14 effects command and control of 
other devices 14 over the network, by supervising software 
and hardware in the controller device 14 such as the XML 
parser 74, the interface fetcher 78 and the XMLRPC codec 

45 76. When a device 14 is a controlled device, a controlled 
application program code 84 in the controlled device 14 
supervises software and hardware in the device 14 for the 
device 14 to be controlled by other devices 14, A Home 
Network Device Web server 86 in each of the devices A and 

50 B manages communication between the devices over the 
network. An XML to Native Lookup Table 88 in each of the 
devices A and B is used by the controlled application 84 to 
convert information in XMLRPC messages (e.g., method 
name, parameters name and type) to native interface of the 

55 device (e.g., native method name, parameters name and 
type). Said table 88 is not used when the names of methods 
and parameters in XML messages and the native interface of 
the device are the same. 

Each of device the devices A and B further includes one 

60 or more Handlers 90, wherein each Handler 90 includes a 
pointer from within the controlled application 84 to a native 
implementation of one specific device functionality. In most 
devices, native implementations of device functionality 
include binary code at run-lime. The binary code can be 

65 generated from higher level languages at compile time, 
including C and Java, for example. As such, consumer 
electronics manufacturers can add more Handlers 90 for new 
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functions without affecting existing Handlers and function 
implementations. A hardware service 92 in each of the 
devices A and B includes native implementations of device 
functions. Each of the devices A and B also includes a Native 
Interface 94 which comprises the API of native implemen- 
tation of the device functions. 

Further, a Network Object Request Broker such as a 
Home Network Object Request Broker (HNORB) 79 and 
Interface Library (IL) 80 provides a middleware layer 98 for 
the home network 10. As shown in FIG. 19, the middleware 
layer 98 can be located in a third device 96 or in a separate 
control hub. The HNORB 79 includes a software agent for 
use by one device 14 to discover the existence of other 
devices 14 connected to the network 10. The HNORB 
software agent organizes device names into a naming hier- 
archical tree structure, organizes device interfaces into said 
searchable Interface Library, and provides device interfaces 
to a device requesting interface information. 

The middleware layer, comprising the HNORB 79 and the 
IL 80, can be connected directly to the Internet, such that 
selected home devices can be accessed from outside of a 
local home network 10. The middleware layer 98 in one 
local home network can be connected to the middleware 
layer 98 in other local home networks over the Internet to 
provide an integrated network comprising two home net- 
works 10. In that case, authorized users with the appropriate 
stream encryption can access a DVD changer in the user*s 
primary home, from a TV in the user's secondary home to 
play a video and view it on the TV. 

To use the Interface Library 80, at least one HNORB&IL 
should be running on the local home network 10. More than 
one HNORB&IL may also be available. For example, a 
cable modem, several DTVs, and a central home hub can all 
have their own HNORB&IL software agents. To locate the 
HNORB&IL, a device 14 sends a broadcast message over 
the local home network. The first HNORB&IL to respond to 
the device 14 is utilized by the device 14. Once a 
HNORB&IL is located, the device 14 and the HNORB&IL 
can establish a point-to-point Transmission Control Protocol 
(TCP) or User Datagram Protocol (UDP) connection for 
registration, interface request and fetch, and device lookup 
services. If a UDP protocol is not available, a TCP protocol 
can be used for high bandwidth connections such as IEEE 
1394. HTTP-based XMLRPC can also be utilized for device 
to HNORB&IL communications. For example, a device 14 
can remotely call a "register" method of HNORB to pass the 
device interface as one or more parameters, or, a XMLRPC 
call can retrieve a partial or entire device interface from the 
IL as a XMLRPC response or return value. 

As aforementioned, more than one HNORB&IT^ can run 
in a local home network 10 simultaneously, wherein each 
HNORB&IL recognizes a subset of available devices and 
one HNORB&IL can communicate with other 
HNORB&ILs to locate the devices 14 it can not find. 
Multiple HNORB&ILs on one local home network 10 can 
locate each other automatically by using broadcasting 
messages, such as UDP or TCP. In this case, multiple 
HNORBs construct a distributed object request broker, 
while multiple Interface Libraries 80 construct a distributed 
interface library. To provide fault tolerance, if one of the 
HNORB&IL should terminate unexpectedly, all devices 
registered with this HNORB&IL are notified and said 
devices can automatically register with another available 
HNORB&IL. 

Each device interface has an associated consistent, unique 
logical name. Other devices can use said consistent, unique, 
logical name to recognize and access a device, even after 
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said device's location or real network address has changed. 
The mapping of the logical names and real device addresses 
are handled by a software agent for naming service in 
HNORB. Preferably, a standardized naming method is uti- 

5 hzed. More preferably, a hierarchical naming structure is 
used to organize device names into a hierarchical tree. This 
hierarchical structure can be expressed using "/", similar to 
that in a file system. The structure can be generated by 
different methods, such as by different service types as a 

10 home/MPEG2/TV; or by different locations, such as home/ 
hvingroomA^CR. Several naming trees may coexist for 
performance and efiBciency. 

In the example command and control between the con- 
troller server device A and the controller server device B in 

15 FIG. 19, the middleware layer 98 is in the third device 96 or 
can be in a separate central hub. The grayed blocks show the 
device elements used for the specific command and control 
process depicted in FIG. 19. In an example operation 
scenario, after the devices A and B become available and 

20 accessible over the network, each device registers/submits 
itself and its XML interface to the central HNORB and IL 
middleware layer 98. If a central HNORB and IL middle- 
ware layer is not available, then each device broadcasts a 
message over the local home network to announce itself. 

25 The controller application 82 of the device A attempts to 
query all or part of the device interface of the controlled 
device B. If an Interface Library 80 is not available, the 
controller device A can request and fetch the device interface 
of the controlled device B directly from the controller device 

30 B by first sending a request to air device B over the network, 
and then receiving the XML interface of device B from the 
device B. However, if an Interface Library 80 is available, 
the controller device A can request all or part of the device 
interface of the controlled device B from the Interface 

35 Library 80. The software agent of HNORB obtains the XML 
device interface of the device B from the Interface Library 
80 structure and sends it back to the controller device A. 

Once the controller device A receives the XML device 
interface of the controlled device B, the controller applica- 

40 tion of device A uses the XML parser 74 of device A to parse 
and interpret the device interface of the device B. The 
XMLRPC codec 76 of device A then generates desired 
XMLRPC command messages using the parser results. The 
XMLRPC command messages are sent to the controlled 

45 device B over the network. Upon receiving said XMLRPC 
command messages, the controlled application 84 of device 
B uses the XML parser 74 of device B to parse and interpret 
the received XML command messages. The XMLRPC 
codec 76 of device B then decodes the parser results to 

50 obtain the method call information in the command 
message, including a method name and parameters for the 
device B functions to perform requested services. 

The controlled application 84 of device B then uses the 
XML to Native Lookup Table 88 and Handlers 90 in the 

55 device B to access and launch the native function imple- 
mentations of device B through the native interface of 
device B. If a function generates any responses or return 
values, said responses or return values are encoded into 
XML or XMLRPC messages and sent to the controller 

60 device A. Further, the middleware layer HNORB and ILcan 
provide the controller device A with a reference to the 
controlled device B, whereby the device A can generate 
remote calls to the device B native functions just as calls to 
the local device A native function. 

65 Preferably, a standard XMLRPC format is utilized so that 
all devices can interpret and decode RPC calls over the 
network. Because the device interface of a controlled device 
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14 can be queried and examined by a controller device 14, 
preferably a simplified XMLRPC format with sufiBcient 
device interface information is utilized to improve effi- 
ciency. The following example shows two possible formats 
of XMLRPC calls for One Touch Record (OTR) and Time 5 
Delayed Record (TDR) operations. 

EXAMPLE I 

XML RPC call, example formal including detailed tag 
and interface information: 

1, Example of OTR call: 
<?xml version="1.0"?> 
<call> 

<obj ect> D VCR 1 . record</object> 

<mcthod>timeDelayedRecod</method> 

<parameters> 
<pararaeter> 

<name>channel</name> 

<value><inl>4</int></value> 
</parameter> 
<parameter> 

<name>recordTime</name> 

<value><time>2:10:30</time><A'alue> 
</parameter> 

</parameters> 
</call> 

2. Example of TDR call: 
<?xml version="1.0"?> 

<call> 30 

<object>DVCRl.record</object> 

<method>oneTouchRecod</method> 

<parameters> 
<parameler> 

<name>channel</name> 

<value><chaonelName>NBC</channelName></ 
value> 
</parameter> 
<parameter> 

<name>startTime</name> 

<value><datetimc.iso8601>1999040lT19:05:35</ 
datetim 

e .iso860 1 ></value > 
</parameter> 
<parameler> 

<name>recordTime</Dame> 

<value><lime>2:00:00</time></value> 
</pararaeter> 

</parameiers> 
</call> 

EXAMPLE 11 

XML RPC call, example format with reduced tags and 55 
interface information: 

1. Example of OTR call: 
<?xml version«"1.0"?> 
<call> 

<object>DVCRLrecord</object> 
<method>timeDelayedRecod</method> 

<parameter value •■"4">channel</parameter> 
<parameter value="2:10:30">recordTime 
</parameter> 

</call> 65 

2, Example of TDR call: 
<?xml version«"1.0"?> 
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<call> 

<object>DVCRl.record</object> 
<method>oneTouchRecod</method> 

<parameler value«"NBC">channel</parameter> 

<parameler 

value«"19990401T19:05:35">startTime</ 
parameter> 

<parameter value="2:00:00" >recordTime</ 
parameter> 

</call> 

Referring to FIG. 20, device interfaces for home devices 
14 are based on an industry standard structured data base 
100 using standardized vocabulary. Interface data for new 
interfaces and vocabulary can be added to the data base 100. 
A comprehensive definition or database of CE objects, 
methods and parameters using English words to describe all 
CE devices is termed a CE data base 102. The comprehen- 
sive definition or database can be in C, XML or other 
formats capable of representing objects and their respective 
methods and parameters. The comprehensive definition or 
database utilizing standardized XML vocabularies is termed 
XCE definition or data base 104. 

Controller and controlled applications 82, 84 are pro- 
grammed using a standard interface subset of the XML 
based XCE data base 104. Each device interface is stored 
with said applications 82, 84 in XML form. Although the 
XCE data base 104 need not be in XML, said subset 
interface produced at compile time is in XML in an embodi- 
ment of the invention, as described above in reference to 
FIG. 15. 

In FIG. 20, for embedded appliances 14, the information 
designated as 'Manufacturer* information is built-in to the 
appliances 14 at manufacture time, and the information 
designated as 'Home Network' is part of the operational run 
time aspects of the appliance in the network. Device XML 
interfaces 72 designated as I ... N for N devices 14, are 
branches of the data in a standardized XCE data base 104. 
A Home Network Interface Library (HNIL) 106 provides a 
collection of the device interfaces of available devices 14 
connected to the home network. The Home Network Inter- 
face Library 106 is a subset of the totality of the XCE data 
base 104. 

In FIG. 16, a device interface was transferred from a 
device A to a device B for an Application B in device B to 
examine the contents of the interface for the device A. As 
detailed above, a device interface includes a description of 
the objects, methods, parameters supported by a device, and 
is referred to as INTERFACE-A.XML for a device A for 
example. A Device XML interface 72 is a device interface in 
XML format. The content of the XCE data base 104 is a 
service oriented structure which provides device interfaces. 

Referring to FIG. 20, the XCE database 104 also includes 
a standardized XCE Interface Document Type Definition 
(DTD) for CE devices, which provides a standardized set of 
rules for i^ing XML to represent CE devices 14. The DTD 
or its subsets can be used for validity checks. A software 
agent designated as Manufacturer Tool 108, filters and 
utilizes a subset of the standardized XCE definition 104 for 
a specific CE device, and uses the standardized XCE Inter- 
face DTD to generate an XML device interface 72 of the CE 
device, for example INTERFACE.XML and INTER- 
FACE.DTD. The document INTERFACE.XML includes a 
description of the objects, methods and parameters sup- 
ported by a specific device according to the standardized 
XCE Interface DTD. The document INTERFACE.DTD is a 
subset of the standardized XCE Interface DTD, and can be 
used for validity check for the XML interface of the device. 
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Other document type definitions can also be used to create 
the INTERFACE.XML document. 

The XML interfaces 72 of the CE devices, including said 
XML interface document and said DTD document, are 
stored in a universally accessible library such as the home 
network Interface Library 106. Asoftware agent 110 collects 
the device interfaces 72 of all accessible devices 14 over the 
network and places them into the searchable structured 
Interface Library 106 along with the device name/address 
information. The Interface Library 106 is a subset of the 
XCE database 104 and the process of generating the Inter- 
face Library 106 is similar to that of rebuilding part or all of 
the XCE database 104. The Interface Library 106 can 
function as a collection of device interfaces 72 of all devices 
14 in the home network, or as a cache depending on 
availability of storage space, wherein only the most recently 
used device interfaces 72 are stored therein. In cases where 
a device 14 updates its device interface 72 due to an event, 
such as disk change in a DVD player, part of the device 
interface 72 is updated based on an event service. 

Referring to FIG. 21, preferably the device interface 
definition 72 of each device 14 has a hierarchical form. This 
is because for a home device 14, the device interface 
definition 72 can be lengthy. Typically, one or few functions 
such as a single function for Time Delayed Recording, are 
accessed at a time, and therefore only a small portion of the 
device interface 72 is used. Rather than rendering the entire 
device interface 72, it is more efficient to render only a 
portion of he device interface 72. By using hierarchical 
device XML interface, a controller device 14 can request 
partial device interface 72 of a controlled device 14 by 
specifying the desired function categories or functions in a 
request for the XML device interface from the controlled 
device 14 or from the HNORB and IL middleware layer 98. 
In the latter case, the HNORB and IL middleware layer 98 
sends back the desired portion of the device interface 72. 

Referring to FIG. 21, the hierarchical device interface 
structure can include four layers, including: (1) a first layer 
112 for XML interface of each home network, listing current 
available devices, (2) a second layer 114 for general XML 
interfaces of each device, listing function categories, (3) a 
third layer 116 for specific XML interface of each function 
category for a device, and (4) a fourth layer 118 for specific 
XML interface of each function in a function category. 
Inside the home network, only the three lower layers 114, 
116 and 118 are utilized, and outside the home network the 
first layer 112 is utilized. 

FIG. 22 shows said layers 112, 114, 116, 118 and corre- 
sponding interface examples. The interface in each layer is 
linked to upper or lower layer (if available) through links 
such as XLink or XPoinler, which provide two-way linking. 
XLink includes a package of hyperlinking functionality that 
has two parts: (1) an XLink component which allows links 
in an XML documents to be recognized as such, and (2) an 
XPointer component which allows links to address into 
precise sub-parts of an XML document. As such, XLink 
governs how links are inserted into XML documents, 
wherein the link may point to data such as a GIF file. 
Further, XPointer governs a fragment identifier that can go 
on a URL when linking to an XML document, from any- 
where (e.g., from an HTML file). 

In a typical command and control model for a server 
device 14 to control another server device 14 according to 
the present invention, a first device 14 attempts to query the 
device interface of a second device 14 at the second interface 
layer 114. After selecting function categories (FC), the first 
device 14 queries the interface layer 116 of a specific 
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function category in the second device 14, such as Record 
Category. Further, the first device 14 can query the interface 
layer 118 of a specific function, such as OTR or TDR, to 
make calls to said functions. The hierarchical or tree struc- 
ture makes finding of an interface function more efficient 
and saves network bandwidth. An example interface file 
strucmre and layers can be: 
First layer 112— HNLxml 
Second layer 114— VCRl.xml 
Third layer 116— VCRl__RecordCategory,xml 
Fourth layer 118— VCRl_RecordCategory_OTR.xml 
Similarly, the home network Interface Library 106 is 
preferably hierarchical and can be structured in a variety of 
ways such as by different service type of devices or by 
different locations such as rooms. Said hierarchical structure 
is the interface of a local home network 10 to other home 
networks or the Internet. 

Appendix 5 shows an example hierarchical device inter- 
face definition 72 which can be implemented in XML 
syntax. Said hierarchical device interface definition 72 can 
include the following fields: 

'document file' name, provides name of the document 
type definition (DTD) file that can be used by an XML 
parser 74 for verification of legality and correctness of 
the XCE database 104 or part of the XML version of the 
XCE database 104. lliere can be several DTD files for 
different parts of the XCE structure, wherein said DTDs 
are different from the document type definitions for the 
RPC.CALL and INTERFACE.DTD for communica- 
tion. 

'doc' name, provides the top level name of the area of 
coverage of capabilities, attributes, communication and 
control interface. 

'Services_home*, provides area for home automation, 
consumer electronics, utility, etc. 

*Server_auto', for an automobile in the garage and shows 
message interface available for one or more automobile 
types. For example, server_auto__ford_explorer_98* 
is the interface for a particular automobile. This allows 
access to mileage and maintenance interfaces of the 
automobile, and can also be used for remote access by 
an automobile manufacturer or garage for direct check- 
ing and remote diagnostics, for example, 

'server_samsung_web_sile', provides for communicat- 
ing with a manufacturer Web site outside the home. 
Includes interface for message, service, help, etc. 

'AVC_commands' and 'CAL„commands', provides for 
legacy devices capable of interpreting AV/C and CAL 
languages, for example. This portion of the structure 
identifies commands in said languages, where the com- 
mands are tagged and carried in XML, As such, the 
contents are not XCE (Web) objects, and protocol 
converter applications can be utilized to interface to the 
original CAL or AV/C application software. 

In the above description, *Services_home' provides the 
main structure including AN consumer electronics. A branch 
of the structure is expanded in detail for a particular example 
of a video services sink, and stream destination (e.g., 
DVCR) control interface. The control interfaces in a typical 
home network can include: 

'xml_utility', provides details for supporting utility net- 
work functions such as downloading an updated DTD 
file, interface file, program file, etc. 

'client', describes the interface details of a client device 
12 including a Web browser. For example 'acknowl- 
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edgment' indicates the controller's acceptance of addressed by the lower layers of the definition struc- 

acknowledgraent of a message or command sent out. ture. 

'server_av', provides control and capacity interfaces for 'source' provides interface similar to the 'sink* interface, 

all audio and video services available, including STB, Referenced above, 'service_id' or 'application. 
DVCR DTV DVD AUDIO etc. ^ interface_id* includes the name, address or Web address or 

' ' ' ^ ' u • URL location of one or more devices 14. Because the XCE 

•lighting , provides an interface to a home automation ^^^^^^^^ comprises the totality of agreed upon 

lighting controller, and includes sensors, lights, etc. interfaces, typically a Dynamic Host Configuration Protocol 

'comms', provides control interfaces to communications (DHCP) software agent executes to assign an address and a 

devices, typically for utility purposes or remote man- default name to each device, and the address and a default 

agement of the devices* set-up or parameters, or for name are added the interface of the service or device. The 

restoring configurations. software agent 110 then collects device interfaces 72 which 

•hvac', provides interfaces for. remote control of the include subset or 'device partial XCE* definitions from all 

HVAC system, and can be used for control of said ^^e devices locally connected to the home network to 

system from outside the home by the utiUty company, 35 J P^^^f^ . Additional relevant exter- 

/ 1 . . .1. u » n\/Ar^ . nal interfaces can be added to the structure for external 

for example, to turn the home s HVAC system oft * 1 i- i * -j* u / jj • 

„ r- i- J control, r or example service_id can be a name/address m 

during peak load penods of he day. Further, said ^ ^^^.^^^ ^^^^^/^^ ^ ^^^^^^^ j^^^^^^^ library 106 

mterface can be used for controUing the HVAC system including entries from the software agent according to the 

from within the home, by an apphance for device based ^^^^^^ interfaces of the devices connected to the network, 
controller to provide a more sophisticated control 20 Thereafter, a user can search for a service in the database and 

mechanism than thermostat control. access an application whose interface includes a particular 

'utility*, provides interface for reading utility meters for data branch of the library using said name/address. As such, 

the home, for example. the network can include multiple identical services distin- 

'security', provides interface for security sensors and guished by said name/address information, 
alarm settines 25 'media , provides interface for the type of media 

As such, using the interface, appUcations runoing on a home w"'°rn''"' n^"" ' aT'' ^ 

, \t • u . J ^ . » from a PC DRAM, disk for CD or DVD, and tape. The 

network device can have access to the sensor and detector ^^^.^ ^^^^ ^ controller device 

devices around the home for monitoring and controlhng of ^^^^^^ ^CE data base to identify the media currently 

the those devices. provided on the network. When a new media such as DVD 

'appliances*, provides interfaces for kitchen, utility and disk is provided on the network, that portion of the device 

general home appliances, including, for example, pro- to interface 72 identifying the program material on the disk 

viding remote control or monitoring temperature set- is changed accordingly. As such, the entire device interface 

tings or other controls and parameters from a controller 72 need not be transferred and only the relevant portion is 

device. In one scenario, a microwave appliance can transmitted to the XCE data base. On receipt of an attention 

scan bar code information on the packaging of a food signal, the library software agent 110 can fetch the new 

item and access a manufacturer database lo obtain update and place it in the proper place in the interface library 

cooking time of the food for a given microwave system The addition of the disk media is similar to adding a 

type. Such integration of appHances using device to service to the network or connecting another appUance to the 

device command and control provides many control network. , ^ , ^ ^ ■ 

scenarios for providing services such as automaticaUy '° . '"j* ' P^'^'d^* " Y ''^'^T-,'TJ°' ^ 

pausing a dishwasher and muting a TV when a phone ^ Mbits/Sec or 19.2 Mb.ts/Sec, for 

is picked up in the kitchen or family room. examp ^- 

. . . , 'protocol , identifies the protocol used for said data 

'convenience*, provides mterfaces to devices for provid- ^^^^^^ ^^^^ ^^^^^ ^^^^ provided, for example 

ing convenience services such as interface to a curtain, 6I883/1394 or UDP/IP, then a desired protocol can be 

window, blinds or whirlpool controllers, for example. selected 

In the above description. 'seiver_av* is part of the stnic- 'stream_format*, provides packet format and/or com- 

ture for the control interfaces for AA^ apphances offermg ^^^^^^^^ ^-^j^j ^^^^^^ ^^^^ ^^^^ j,p,it. if 

A/V stream service, and is subdivided mto controls-gen , ^^^^ ^^^^ ^^^^ ^ provided, a desired format can be 

'source* and 'sink' capabilities. ^^j^^^j ^„ interface message. A controller application 

'controls-gen*, provides interface for device manufacturer 82 can examine the available formats to determine if there 

attributes and general utility interfacing such as ping are compatible ones. 

testing the presence of the device. Further, *controls__av', provides the main control interface for 

manufactured-in attributes such as software and hard- a/N media appliance. 

ware identification and version information can also be 55 •Flow_contror, provides data stream controls such as: 
included. Adevice supplying this interface returns data PLAY, STOP, GOTO, RECORD, etc as methods for a 
providing name or identification for said software with- particular device. The methods do not change for embedded 
out effecting any control actions. An interface to set the appliance, except for PC software, for example. The controls 
time of day clock can also be included. can include time parameters for delayed operation, 
'sink*, provides interface for the media stream service 60 'Tuning*, provides interface for tuning control. A control- 
devices. The structure is organized based on service ler device 14 can send a request to the interfaces of a 
offered (i,e. video stream record and replay) rather than controlled device 14 to send back an Electronic Program 
particular device names such as VCR. For example, a Guide (EPG) data structure described above. 
Tuner and a DVD player are both video program stream *UI control*, provides control interface to a controlled 
sources for the network with video program formats, 65 application 84 to control adjustments for display such as 
and can be controlled, such as started and stopped. brightness and contrast, and for audio, such as volume and 
Differences in control of particular devices are bass. 
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Timcr_record* provides interface for set-up data for a 
controller application 82 to implement delayed time record- 
ing. Direct channel tune information and flow control 
(time_aparams) information can be utilized. 

The above description can apply equally to client devices 
12. An alternative syntax XCE definition or database for the 
CE space can be utilized. The alternative syntax XCE data 
base includes full service descriptions including home 
automation, appliances and automobile, for example. In 
cases where a service object provides flexibility and param- 
eters for control, a control method is utilized to control the 
object as desired. Appendix 3 shows example commands in 
the AV/C and CAL command languages including binary 
and hex data strings. 

In another aspect, the present invention provides for use 
of existing command language implementations for device- 
to-device command and control in a network. Devices can 
include internal objects and APIs which, at run time, create 
binary strings according to existing transport mechanisms. 
In that case, in order to provide XML remote procedure caUs 
(XML RPC) from one device 14 to another device 14 in the 
network, the exiting application interface implementation is 
replaced with calls to the XML service API. As such, the 
original implementation is equivalent to a wrapper for the 
XML service API. FIG. 18 also shows applications created 
using other command languages such as CAL or AV/C in 
dashed lines, with their interface implementations replaced 
with a wrapper in the XCE/XML service API. Appendix 4 
shows examples for changing from CAL command language 
to XML RPC format. 

Referring to FIG. 23, in another aspect, the present 
invention provides a standard command protocol and control 
language translation for inter-device communication 
between disparate devices in a network. For different 
devices to share information, the information must be in a 
format that a requesting device can interpret. And, for a 
device 120 to control another device 22, the two devices 
must use a common language in order to interpret one 
another*s commands. The present invention provides a com- 
mon identification format for data and command protocols. 

In one embodiment, a method of common presentation or 
packaging of data and command protocol is provided, 
whereby a receiving device 122 can determine the native 
format of transmitted data. If the receiving device 122 can 
interpret that native format, then it can accept the data 
directly. Otherwise, the receiving device 122 can request a 
translator device 124 or application to translate the data into 
a desired formal which the requesting device 122 can 
interpret. The translator device 124 or application deter- 
mines the native format of the original data, translates the 
data into said desired format, and sends the translated data 
to the requesting device 122. 

The requesting device 122 then processes that data as 
though the data had originally been provided in the request- 
ing device's native language format by the sending device 
120. The requesting device 122 can also send a response 
back to the sending device 120 in the requesting device's 
native format, or send a response by proxy through the 
translator device 124 or application for translation into the 
native format of the sending device 120. The translation 
method can be utilized for information including command 
protocols, data files and audio/video streams. 

For devices that do not utilize the common format 
described above, the present invention provides for transla- 
tion of data including command protocols to, and from, such 
non-compliant devices. For example, when a non-compliant 
device 120 sends data to a compliant device 122, the 
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compliant device 122 can translate the data based on a 
determination of the native format of the data. For example, 
the compliant device 122 can examine the data for particular 
bit patterns within the data. When a compliant device sends 
5 data to a known non-compliant device, the compliant device 
can translate the data before transmission based on a deter- 
mination of the native format of the non-compliant device. 

An example implementation can be for a home network 
which supports the IP and HTTP protocols. The home 
network can be connected to the Internet to obtain applica- 
tions and services of various types for desired functionality. 
As such, the common format method can be made compat- 
ible with Internet protocols and procedure for operation over 
the Internet and the home network. 

One example of providing a common data format is 
utilizing XML to create a package for the data for transmis- 
sion over the home network. The data can include command 
protocol, streaming audio or video, graphics or apphcations. 
The data is ^wrapped* with a standard header identifying the 
native format of the data and contents of the package, in 
XML form. The header allows unique identification of the 
data type the data portion of the XML code, whereby the 
data can be translated if necessary and provided to appro- 
priate applications upon receipt, 
2j Under the Web standard, the identification process is 
performed by browsers using file name extensions to iden- 
tify the type and contents of a file transmission. The brows- 
ers then launch appropriate plug-in modules to process the 
file. In the home network, XML is utilized to identify data 
transmissions which provides all home network transmis- 
sions over IP with a common identification method as 
described above. 

Alternatively, a software layer can be provided in the 
home network protocol stack to uniquely identify the con- 
tents of all data transmissions over the home network. The 
software layer can be used instead of XML. The common 
format and identification principles of the present invention 
apply equally in either embodiment using XML or said 
software layer as identification methods. 

In FIG, 23, upon receipt of a data package transmission, 
the receiving device 122 examines the XML identity header 
of the data package to determine the format of the data 
therein. If the data is in a format recognizable by the device 
122, the XML identity header information is discarded and 
the device processes the data directly. Otherwise, the device 
122 converts the received XML package into an XML 
translation request package and sends the request package 
and the data to the translation server device 124. 

The translation server device 124 translates the data and 
converts the translated data into an XML translation 
response package. The translation server 124 then transmits 
the response package back to the requesting device 122. In 
case of a translation error, the translation server 124 can 
provide a translation response error condition to the request- 
ing device 122. Upon receiving the translated data, the 
requesting device 122 processes the translated data in the 
response package. 

Example of an XML data package or packet can be: 

<IDENTITY type«»format=AV/c>. . . packet data . . . 
60 <\IDElSnTTY> 

Example of a translation request package or packet can 
be: 

<TRANSLAnON REQUEST type=Command fora3at« 
CAL> 

65 <IDENTITY type=Command formatoAV/C>. , . packet 
data . . . </IDEOTlTY> 
<\TRANSLATION REQUEST> 
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Example of a translalion request package or packet can 

be: 

<TRANSLAnON RESPONSE typeoCommand formato 
CAL>. . . packet data . . . 

<\TRANSLAnON RESPONSE> 5 

Example of a translation response error condition package 
or packet can be. <TRANSLATION RESPONSE type« 
Command formai=CAL>. . . packet data . . . 

<ERROR condition«Unrecognized 
command>Translation could not be lo 
performed <|ERROR> 

<\TRANSLAnON RESPONSE> 

Further, Table 3 in FIG. 24 includes a partial list of 
package or packet types and formats. 

To provide translation services, a translation server 124 is 15 
identified in the network during network configuration in a 
manner similar to that of DHCP servers. The translation 
server 124 broadcasts its IP address to all devices in the 
network for a period of time after the network is configured. 
All devices 120, 122 compatible with the translation ser- 20 
vices store the IP address of the translation server 124 as it 
is broadcast over the network during network boot up. 

Alternatively, the requesting device 122 can broadcast a 
translation request over the home network. All translation 
servers 124 in the network that receive the translation 25 
request can respond to the translation request by sending a 
translation response to the requesting device 122. The 
requesting device 122 then selects one translation server 124 
among the responding translation servers. In one example, 
the requesting device 122 selects the first translation server 30 
124 that responds to the translation request. In another 
example, the translation servers 124 can negotiate among 
themselves and/or with the requesting device 122 for the 
selection of a translation server 124 for satisfying the 
translation request, 35 

In another embodiment of the invention, muUiple trans- 
lation servers 124 are utilized to fulfill all translation 
requests. For example, a single translation server 124 may 
not have the capabiUty to translate all requests. In such 
cases, it is necessary to identify the address of each trans- 40 
lation server 124 and the type of translation service each 
translation server 124 can provide. Each device 120, 122 can 
store a list of all translation server IP addresses and a 
corresponding list of the types of translation services each 
translation server 124 provides, and optionally the associ- 45 
ated translation application. 

For efiSciency, if a sending device 120 wishes to send data 
to a receiving device 122 which is known to use a different 
native format than that of the sending device 120, the 
sending device 120 can send the data to the receiving device 50 
122 by proxy through a translation server 124. The sending 
device 120 transmits a command to the translation server 
124 similar to the translation request command and includes 
the address of the receiving device 122 as the destination for 
the translated data. 55 

In cases where a receiving device 122 requires translation 
of a data stream, the sending device 120 can route the data 
stream directly to a translation server 124, and the transla- 
tion server 124 in turn transmits the translated data to the 
receiving device 122 as described above. Alternatively, the 60 
sending device 120 can send the data stream lo the receiving 
device 122, and the receiving device 122 then routes the data 
stream to the translation server 124 for translation and return 
of the translated data back to the receiving device 122. 

In the description herein, the control mechanism is based 65 
on the Hypertext Transfer Protocol (HTTP 1.1) which pro- 
vides an application-level protocol for distributed, 



collaborative, hypermedia information systems. HTTP is a 
generic, stateless, object-oriented protocol in wide use for 
many tasks. A feature of HTTP is the typing and negotiation 
of data representation, allowing systems to be built inde- 
pendently of the data being transferred. Preferably, the 
network protocol used by devices and applications on the 
home network is IP (Internet Protocol). However, other 
protocols can also be utilized. 

Although the present invention has been described in 
considerable detail with regard to the preferred versions 
thereof, other versions are possible. Therefore, the appended 
claims should not be limited to the descriptions of the 
preferred versions contained herein. 

What is claimed is: 

1. A method for command and control among a plurality 
of devices via a network, the method comprising the steps 
of: 

(a) connecting a first device to the network; 

(b) connecting a second device to the network, the second 
device storing appHcation interface description data for 
commanding and controlling the second device by at 
least one other device connected to the network; 

(c) receiving said application interface description data 
from the second device over the network; and 

(d) sending control and command data from the first 
device to the second device over the network utilizing 
said application interface description data to control the 
operation of the second device. 

2. The method of claim 1, wherein step (c) further 
includes locating said application interface description data 
via the network, and providing said application interface 
description data to the first device via the network. 

3. The method of claim 1, wherein: 

(i) step (b) includes connecting two or more devices to the 
network, each device storing application interface 
description data in said structured format for command- 
ing and controlling of the device by one ore more other 
devices connected to the network; 

(ii) step (c) includes providing the application interface 
description data of a plurality of said devices to the first 
device via the network; and 

(iii) step (d) includes sending control and command data 
from the first device to said plurality of devices over the 
network utilizing the application interface description 
data corresponding to each of said plurality of devices 
to control the operation of said plurality of devices. 

4. The method of claim 3, wherein step (ii) further 
includes locating said application interface description data 
via the network, and providing said application interface 
description data to the first device via the network. 

5. The method of claim 3, wherein step (ii) further 
includes providing the application interface data of a plu- 
rality of said devices to at least the first device, and wherein 
step (iii) includes sending control and command data from 
at least the first device to said plurality of devices connected 
to the network utilizing the application interface description 
data corresponding to each of said plurality of devices to 
control the operation of at least one of said plurality of 
devices. 

6. The method of claim 1, wherein step (c) includes 
transferring at least a portion of said application interface 
description data to the first home device via the network. 

7. The method of claim 1, wherein step (c) includes the 
first device querying the application interface description 
data in the second device via the network, 

8. The method of claim 1, wherein step (c) includes the 
first device querying the application interface description 
from a database device connected to the network. 
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9. The method of claim 1, wherein the application inter- more controlled device utilizing said application interface 
face description data includes remote procedure call infor- description data to control the operation of said one or more 
mation for the first home device to control the operation of controlled devices. 

the second home device. 23. A method for performing a service via a home 

10. ITie method of claim 1, wherein the application 5 network, the method comprising the steps of: 

interface description data includes capabilities data for iden- (a) connecting a client device to the home network, 

tifying the capabilities of the second device. wherein the client device is capable of displaying user 

11. The method of claim 1, wherein said devices are interface data; 

incapable of displaying user interfaces. (b) connecting a first home device to the home network, 

12. Tlie method of claim 1, wherein said application jq the first home device storing user interface data in a 
interface description data includes XML format. selected format that defines a user interface for user 

13. A network system for commanding and controlling command and control of at least the first home device 
devices, comprising: by a user via the network; 

(a) a physical layer, wherein the physical layer provides a (c) connecting a second home device to the home 
communication medium than can be used by devices to 15 network, the second home device storing application 
communicate with each other; interface description data in a structured format for 

(b) at least one controlled device containing application essentially autonomous device command and control of 
interface description data in a structured format for the second home device by one or more other home 
commanding and controUing the controlled device by devices connected to the network; 

at least one other device; and 20 (d) receiving the user interface data of the first home 

(c) at least one controller device including a device device at the client device via the home network; 
control for obtaining said application interface descrip- (e) accepting user input from a user in response to the user 
tion data, and sending control and command data to the interacting with the user interface; and 

controlled device utilizing said application interface (f) sending control and command data from the client 

description data to essentially autonomously control 25 device to the first home device based on the user input 

the operation of the controlled device. to cause the first and second home devices to essen- 

14. The network system of claim 13 further comprising a tially autonomously communicate with each other uti- 
pluraliiy of controlled devices, each controlled device stor- lizing said application interface description data to 
ing application interface description data for commanding perform the service. 

and controlling of each controlled device by at least said 30 24. The method of claim 23, wherein step (f) further 

controller device, wherein said device control selectively includes accepting user input for selecting the second home 

obtains the application interface description data of one or device from the user interface being displayed on the client 

more of said controlled devices for sending control and device. 

command data to one or more of said controlled devices 25. The method of claim 24, further including the step of 

utilizing said application interface description data to control 35 the first home device controlling the second home device by 

the operation of said one or more controlled devices. sending control and command information to the second 

15. The network system of claim 13 wherein said device home device utilizing said application interface description 
control obtains said application interface description data by data based on the user input to the first home device via the 
transferring at least a portion of said application interface client device. 

description data to the controller device for generating and 40 26. The method of claim 25, further comprising the step 

sending said control and command data to the controlled of providing the application interface description data to the 

device. first home device via the network. 

16. The network system of claim 13 wherein said device 27. The method of claim 23, wherein said application 
control obtains said application interface description data by interface description data includes capabilities data for the 
querying said application interface description data in the 45 second home device, and further comprising the steps of: (i) 
controlled device. querying the capabilities data within the application inter- 

17. The network system of claim 13 wherein said device face description data of the second home device, and (ii) 
control obtains said application interface description data by updating said user interface data in the first home device 
querying the application interface description data from a using the capabilities data to allow commanding and con- 
database device. so trolling of the second home device by a user via the user 

18. The network system of claim 13 wherein the appli- interface of the first home device displayed on the client 
cation interface description data includes remote procedure device. 

call information for the controller device to control the 28. Tht method of claim 23, farther comprising connect- 

operation of the controlled device. ing two or more home devices to the network, each home 

19. The network system of claim 13 wherein the appli- 55 device storing application interface description data for 
cation interface description data includes capabilities data commanding and controlling of the home device by one or 
for identifying the capabilities of the controlled device, more other home devices connected to the network. 

20. The network system of claim 13 wherein said con- 29. The method of claim 28 wherein the application 
troller and controlled devices are incapable of displaying interface data in each of said two or more home devices 
user interfaces. 60 includes capabilities data for the respective home device, 

21. The network system of claim 13 wherein said struc- and further comprising the steps of: (i) querying the capa- 
tured format includes XML format. bilities data within the application interface data of said two 

22. The network system of claim 13 further comprising a or more home devices, and (ii) updating said user interface 
plurality of controller devices, each controller device includ- data in the first home device using said capabilities data to 
ing a device control for obtaining apphcation interface 65 allow commanding and controlling of said two or more 
description data of one or more controller devices for home devices by a user via the user interface of the first 
sending command and control information to said one or home device displayed on the client device. 
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30. The method of claim 28, further including the step of 
providing the application interface descriptions of a plurality 
of said two or more home devices to the first home device 
via the network. 

31. The method of claim 30, further including the step of: 
sending control and command data from the first home 

device to said plurality of home devices via the network 
utilizing the application interface description data cor- 
responding to each of said plurality of home devices to 
control the operation of said plurality of home devices. 

32. The method of claim 30, further including the step of: 
locating said application interface descriptions over the 

network, and providing said application interface 
descriptions to the first home device via the network. 

33. The method of claim 30 further including the steps of 
providing the application interface description data of one 
home device of said plurality of home devices to another 
home device of said plurality of home devices. 

34. The method of claim 33 further including sending 
control and command data from said one home device to 
said another home device via the network utilizing the 
application interface description corresponding to said 
another home device to control the operation of said other 
home device. 

35. The method of claim 23, wherein the application 
interface description includes remote procedure call infor- 
mation for the first home device to control the operation of 
the second home device. 

36. The method of claim 35, wherein the application 
interface description includes capabilities data for identify- 
ing the capabilities of the second device. 

37. The method of claim 23 wherein the selected format 
includes HTML format. 

38. The method of claim 23 wherein the structured format 
includes XML formal. 

39. A network system for commanding and controlling 
devices, comprising: 

(a) a physical layer, wherein the physical layer provides a 
communication medium than can be used by devices to 
communicate with each other; 

(b) a first server device storing user interface data in a 
selected format that defines a user interface for user 
command and control of at least the first device by a 
user; 

(c) a second server device storing application interface 
description data for essentially autonomous device 
command and control of the second sever device by 
one or more devices; 

(d) a client device capable of displaying user interface 
data, the client device including a user interface con- 
troller for displaying said user interface of the first 
server device on the client device to accept input from 
a user, and for sending control and command data to the 
first server device based on the user input, to cause the 
first and second sever devices to essentially autono- 
mously communicate with each other utilizing said 
application interface description data to perform a 
service requested by the user. 

40. The network system of claim 39, wherein said user 
interface controller accepts user input for selecting the 
second sever device from the user interface being displayed 
on the client device. 

41. The network system of claim 40, wherein the first 
server device includes a device control for controlling the 
second sever device by sending control and command 
information to the second sever device utilizing said appli- 
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cation interface description data based on the user input to 
the first server device via the client device. 

42. The network system of claim 41, wherein said device 
control obtains the application interface description data 

5 from the first server device. ■ 

43. The network system of claim 41 wherein said device 
control obtains the application interface description data 
from a data base. 

44. The network system of claim 41, wherein the appli- 
10 cation interface description data includes capabilities data 

for the second sever device, and wherein the device control 
obtains the capabilities dita from the application interface 
description data and updates said user interface data in the 
first home device using the capabilities data to allow com- 
15 mand and control of the second sever device by a user via 
the user interface of the first server device displayed on the 
client device. 

45. The network system of claim 41 further comprising 
two or more server devices each storing application interface 

20 description data for commanding and controlling of said two 
or more server devices by one or more devices. 

46. The network system of claim 45, wherein the appli- 
cation interface data in each of said two or more server 
devices includes capabilities data for the respective server 

25 device, and wherein the device control obtains the capabili- 
ties data from the application interface data of said two of 
more server devices and updates said user interface in the 
first server device tising said capabilities data to allow 
command and control of said two or more sever devices by 

30 a user via the user interface of the first server device 
displayed on the client device. 

47. The network system of claim 45 wherein the device 
control sends control and command data to said two or more 
server devices utilizing the application interface description 

35 data corresponding to each of said two or more server 
devices to control the operation of said two or more server 
devices. 

48. The network system of claim 39 wherein said appli- 
cation interface description data includes remote procedure 

40 call information for the first server device to control the 
operation of the second server device. 

49. The network system of claim 39 wherein the selected 
format includes HTML format. 

50. The network system of claim 39 wherein application 
45 interface description data includes XML format. 

51. The method of claim 1, wherein: 

the first device includes a controller agent for autono- 
mously controlling one or more devices in the network; 

the application interface description data in the second 
device includes information for use by a controller 
agent to autonomously command and control the sec- 
ond device; and 

step (d) further includes the steps of sending control and 
command data from the controller agent of the first 
device to the second device over the network, wherein 
said controller agent utilizes said application interface 
description information to autonomously control the 
operation of the second device. 

52. The network system of claim 13, wherein: 

said device control in the controller device includes a 
controller agent for autonomously controlling one or 
more devices in the network; and 

the application interface description data in the controlled 
65 device includes information for use by a controller 
agent to autonomously command and control the con- 
trolled device; 
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such that the controller agent of the controller device 
utilizes said apphcation interface description informa- 
tion to autonomously control the operation of the 
controlled device. 

53. The method of claim 23, wherein: 5 
the first device includes a controller agent for autono- 
mously controlling one or more devices in the network; 

the application interface description data in the second 
device includes information for use by a controller 
agent to autonomously command and control the sec- 
ond device; and 

step (g) further includes the steps of sending control and 
command data from the controller agent of the first 
device to the second device over the network, wherein 
said controller agent utilizes said application interface 
description information to autonomously control the 
operation of the second device. 

54. The network system of claim 39, wherein: 

the first server device includes a controller agent for 20 
autonomously controlling one or more devices in the 
network; and 

the application interface description data in the second 
server device includes information for use by a con- 
troller agent to autonomously command and control the 25 
second server device over the network; 

such that the controller agent of the first server device 
utilizes said application interface description informa- 
tion to autonomously control the operation of the 
second server device over the network. 

55. In a network system for command and control of 
devices, including a physical layer having a communication 
medium for communication between controller agents and 
controlled agents in the network system, a control system 
comprising: 35 

(a) at least one controlled agent and application interface 
description information for autonomous command and 
control of the controlled agent by at least one controller 
agent; and 

40 

(b) at least one controller agent configured for using said 
application interface description information to send 



control and command information to the controlled 
agent utilizing to autonomously control the oi)e ration 
of the controlled agent over the network. 

56. ITie network system of claim 55, wherein said apph- 
cation interface description information includes control 
information for use by the controller agent to command and 
control the controlled agent over the network. 

57. In a network system for commanding and controlling 
devices, the network system including a physical layer 
providing a communication medium for communication 
between said devices, a device control system comprising: 

(b) at least one controlled device containing application 
interface description data in a structured format for 
commanding and controlling the controlled device by 
at least one other device over the network; and 

(c) at least one controller device including a device 
control for obtaining said application interface descrip- 
tion data, and sending control and command data to the 
controlled device utilizing said application interface 
description data to essentially autonomously control 
the operation of the controlled device over the network. 

58. The network system of claim 57, wherein: 

said device control in the controller device includes a 
controller agent for autonomously controlling one or 
more devices in the network; and 

the apphcation interface description data in the controlled 
device includes information for use by a controller 
agent to autonomously command and control the con- 
trolled device; 

such that the controller agent of the controller device 
utilizes said application interface description informa- 
tion to autonomously control the operation of the 
controlled device over the network. 

59. llie network system of claim 57, wherein said apph- 
cation interface description data of the controlled device 
includes control information for use by the device control of 
the controller device to command and control the controlled 
device over the network. 
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